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ABSTRACT 

The final work of the National Skill Standards 
Development Project (NSSDP) is to create a model for developing and 
administering a certification program for electronics technicians 
based on already developed skill standards. During its first 2 years, 
NSSDP has developed over 300 standards for work-ready, entry-level 
electronics technicians; developed measurement criteria to assess 
skill mastery; and determined the need for a program to assess the 
skill-based competency of entry-level technicians. The model 
certification program describes a structure and process to create, 
validate, administer, and protect the test and to record test 
results. Two tests have been structured — a technical skill exam and a 
technical fundamentals exam. Proposals from testing organizations for 
contract support in developing a test bank or pool and test forms are 
being reviewed. The test administration process includes these 
components: registration of examinees and verification of 
registrants; establishment of test sites and appointment of proctors; 
development and distribution of test guides; onsite test 
administration; test scoring; and securing of tests and results. 
Issues of data management related to scoring and security have been 
covered. Three options for program management are being considered: 
management by an existing industry organization, establishment of an 
independent test management organization, and assumption of program 
management by an existing independent organization. (YLB) 
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Certification Testing Program 
for 

Entry-Level Electronics Technician Skills 
(National Skill Standards Development Project Final Report) 



I. INTRODUCTION 



A. PHASE I 



The National Skill Standards Development Project (NSSDP) is a 3-year two-phased 
program funded by the U.S. Department of Education’s Business and Education Standards 
Program Initiative. In Phase I, the Electronic Industries Foundation (EIF) and its parent 
organization, the Electronic Industries Association (El A), developed consistent industry-based 
voluntary national skill standards to improve the competency achieved by work-ready, entry- 
level electronics technicians. Over 200 individuals representing over 100 electronics 
companies; government; and national business, labor, and education associations and 
institutions joined in this effort. Their participation was channeled through service on one or 
more of the following bodies: 

• Executive Advisory Board of senior executives, who provided oversight and 
guidance to the project; 

• Technical Committee of mid-level managers, who had direct responsibility for 
developing the skill standards; and 

• Ad Hoc Groups of technicians and their supervisors, who identified skills and 
developed details for the standards. 

• Focus Groups of technicians, managers, trainers, and educators, who 
provided their perspectives on the need for and appropriate content of skill 
standards. 

The standards — over 300 in all — are the industry’s definitive statement of its 
requirements for work-ready technicians capable of entry-level work in any of 1 1 electronics 
specialty occupations: general electronics, avionics, business machine service, consumer 
product service, biomedicine, microcomputer systems, microcomputer systems field service, 
industrial electronics, instrumentation, telecommunications, and automobile service. 
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Following their development, they were validated against workplace performance and 
against standards set by the U.S. Department of Labor’s Secretary’s Commission on 
Necessary Skills (SCANS) and reviewed to assure against bias as to race, sex, or ethnicity. 
They were compared against international benchmarks to assure that U.S. workers would be 
able to meet standards set anywhere in the world. The final standards were approved by the 
project’s Executive Advisory Board and ratified by the EIA Board of Governors. 

The ratified standards were published in manual form in June 1994 as Raising the 
Standard, Electronic Technician Skills for Today and Tomorrow and, as such, were 
distributed to over 3,100 industry and education users and to state directors of vocational 
technical education; the latter were asked to duplicate and distribute the manual within their 
states and territories. 

B. PHASED 

Phase II had two major objectives. The first was to develop measurement criteria by 
which to assess mastery of the skills by entry-level electronics technicians. The criteria were 
developed from original work performed under contract by the Vocational-Technical 
Education Consortium of States (V-TECS) and reviewed and modified by five ad hoc 
working groups of technicians, educators, and supervisors and by the project’s Technical 
Committee. As completed, three or more criteria were identified for each skill standard, 
some with subsections; each set of criteria included the expected results of mastery of that 
skill. The resulting large volume of information led the project to produce the measurement 
criteria on a 3-1/2" PC-compatible disk with the criteria divided into five files keyed to the 
sections and numbering used in the manual. The disk was named Characteristics of 
Competency, Measurement Criteria for Entry-Level Electronics Technician Skills and 
packaged with a cover letter explaining use of the measurement criteria with the standards. 
The packets were completed in October 1995, distributed to the recipients of Raising the 
Standard, and publicized in magazines and through EIA’s World Wide Web site. 

The second Phase II objective was to determine whether industry and electronics 
educators wanted and needed a program to assess the skill-based competency of entry-level 
technicians and, based on an agreement of need, to develop a plan for implementing such a 
program. 

The project conducted extensive research toward that objective: Focus groups of 
educators, working technicians, supervisors of technicians, service providers in the research 
and development, manufacturing, and service sectors, and state directors of vocational 
technical education provided their perspectives on the importance of accreditation of training 
programs, the certification of entry-level technicians, or a combination of both assessment 
options. A literature review and a study of nine accreditation and certification programs 
operated by and for professional and trade organizations created a body of information on 
program components and costs and identified best practices that might be incorporated into a 
program for the NSSDP skill standards. A questionnaire was developed, distributed, and 
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analyzed to obtain industry input on the content, acceptability, and desirability of 
accreditation and certification programs. 

Based on information gathered by these studies, four assessment scenarios (describing 
accreditation, certification, and work and education portfolios) were developed and described 
in another questionnaire distributed to managers and supervisors throughout the industry to 
identify the industry’s preferences. Respondents clearly agreed that assessment of the 
competency of entry-level technicians is important to the industry. They declared a clear- 
cut preference for a certification exam for individual technicians based on the skill standards. 

As a final step to assure that certification would meet industry needs, the project 
surveyed a representative sample of corporate human relations managers industry-wide. 

Their responses clearly indicated that they would view certification as a benchmark in 
determining qualifications for hiring; that they wanted EIA and EIF to sponsor such a testing 
program to assure its integrity and its links to the skill standards; and that access to a list of 
certified individuals would help them identify qualified candidates. 

The remainder of this report completes the final work of the NSSDP: creation of a 
model for developing and administering such a certification program. 



II. A MODEL CERTIFICATION TESTING PROGRAM 

This model certification program, developed following over 2 years of research and 
analysis, describes a structure and process to create, validate, administer, and protect the test 
and to record test results. All sections describe the model as if it had been adopted for use 
("The test will ..." rather than "The test would . . ."). The structure contains four major 
tasks: development of the test itself, development of a process for administering the test, 
development of a data management system, and development of a system for managing the 
program. The report concludes with a brief discussion of the importance of a comprehensive 
marketing program to the success of the certification program. 

A. TEST DEVELOPMENT 

1. Development and Maintenance of Skill Standards 

In order to be valued, any certification test must be based on a body of knowledge 
that the community of users (in this case, educators, technicians, and electronics industry 
employers) agrees is important. This step has been accomplished with the development and 
distribution of the national skill standards for entry-level electronics technicians and 
corresponding measurement criteria described in the introduction to this report. They are the 
industry’s definitive statement of its requirements for work-ready technicians and have been 
accepted by industry and education alike. 




b 



Electronic Industries Foundation — 3 



However, industry requirements change continually as new technologies are 
introduced. It is expected that the skill standards and corresponding measurement criteria 
will be updated periodically to reflect these changes, most likely on a 5-year cycle or as 
technological changes dictate. Successive generations of the test will incorporate these 
updates to assure that the test continues to reflect industry needs. 

2. Determination of the Test Structure 

In meetings with three groups of supervisors of technicians, through questionnaire 
responses, and in focus group sessions, industry expressed its overriding need for a scoring 
system that will give them a profile of the technician’s discrete capabilities in specific 
technical skill areas. That is, they asked for a test that will enable them to determine if a 
candidate’s strengths meet their specific hiring needs. Supervisors preferred a performance- 
based test to a paper and pencil test and expressed the need for some measurement of the 
candidate’s computer literacy and knowledge of the fundamentals of technical 
communications and technical math. 

To best meet these identified needs, two tests have been structured — a Technical 
Skills Exam and a Technical Fundamentals Exam (See Figure 1). The former, covering six 
technical areas, will take 3 to 4 hours to complete; the latter, encompassing three 
fundamental areas, about 90 minutes. A simulation program such as the Electronic Work 
Bench or Electronic Test Bench will substitute for performance testing in at least the test 
equipment and troubleshooting section. The other sections most likely will be paper and 
pencil tests. Simulation testing will eliminate the high costs of training evaluators to obtain 
consistent results and uniform performance at test sites across the country. Examinees will 
be given individual pass/fail scores for each of the six technical areas as well as overall 
pass/fail scores for the Technical Skills Exam and the Technical Fundamentals Exam. 

3. Development of the Test Question Bank and Test Forms 

EIF/EIA is reviewing proposals from testing organizations (e.g., ATC, ETS, etc.) for 
contract support in developing a "bank" or "pool" of test questions and an initial set of test 
forms. All work performed by the contractor selected will be done under supervision of the 
project manager. The contract group will review the test structures described in the 
preceding section and, with project staff, select a final structure. Ad hoc teams of 
technicians, supervisors, and educators will actually write the test questions, working from 
the skill standards and measurement criteria. The contractor will determine the number and 
length of writing sessions needed and the types and numbers of participants, identify sites 
and dates for the sessions, and lead the test writing sessions. Three to five questions will be 
written to cover each discrete skill and piece of fundamental knowledge to be tested. This 
redundancy in the test question bank makes it possible to develop numerous test forms that 
produce the same scoring results and responds to security issues covered later in the report. 
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The contractor will edit the questions for clarity, accuracy, specificity (only one 
acceptable answer), and lack of bias. The Technical Committee will review and provide 
comment, based on which the contractor will create a draft bank of test questions that will be 
validated for use in developing individual certification tests. 

In developing different test forms from the approved and validated bank of test 
questions, the contractor will select among the questions covering like material, make sure 
that questions for each form cover the same technical content at the same depth and that they 
are weighted equally and are of equal merit and difficulty. The number of forms also can be 
increased by putting the same questions in different order to invalidate cheating. After test 
forms have been administered, they will be compared and equated for uniform grading. For 
example, if one form produces consistently lower scores than others, the scores would be 
equated or normalized *o be comparable to the others and the test form would be withdrawn 
and revised. 

Industrial certification tests often are named for the outcome, and a similar approach 
will be taken for the certification test for entry-level electronics technicians. Possibilities 
suggested to date include Basic Electronics Technician (BET), Qualified Electronics 
Technician (QET), and Excellence in Electronics (EIE). 

4. Validation of the Test 



A valid test contains questions unambiguous and unbiased in their presentation; 
produces consistent, repeatable results; and provides a true differentiation between qualified 
and unqualified candidate test takers. The contractor will conduct sophisticated field testing 
to determine that the test meets these criteria. Still working with the project manager, the 
contractor will design the validation process and establish the pass/fail cutoff points for 
scoring; identify the number of validat on sites needed and the types and numbers of people 
to take the field tests to provide the necessary results; provide staff to administer and proctor 
validation tests at selected sites; score and evaluate test results; analyze the results and submit 
a summary of findings to the project manager; and submit to the project manager a final 
report documenting the validation process, results, and recommendations. EIF will arrange 
for actual test sites and individuals to take the tests. 

B. TEST ADMINISTRATION 

Once the certification test is developed, a process must be constructed for 
administering the test. Components of this process will include systems for registration of 
examinees and verification of registrants; establishment of test sites and appointment of 
proctors; development and distribution of test guides and other materials; onsite 
administration of the test itself; scoring the tests; and securing the tests and test results. The 
process has been developed based upon the experiences and comments of the nine 
organizations studied and on a review of the published literature. 
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1. Test Registration and Verification 



To identify and track individuals wishing to take the exam, the following systems will 
be developed: an application/registration form including demographic information about the 
test taker; a method for distributing registration forms in response to direct requests and 
through publications, educational institutions, and worksites; advanced registration so that 
program managers will know how many people will be taking each administration of the test, 
who they are, and which test sites they will use; a means of verifying that the person actually 
taking the test is the person who registered to take it (e.g., photo identification); a method 
for determining appropriate registration fees and collecting them; a process for identifying 
persons with disabilities and persons with language or reading difficulties unrelated to 
disabilities (e.g., speakers of English as a second language) who will need accommodations 
to take the test; and a means of providing those accommodations. 

2. Test Sites and Proctors 

The electronics industry sorely needs both qualified technicians and help in identifying 
who they are. It is to the industry’s advantage to certify qualified technicians as quickly as 
possible. Thus, a network of test sites will be developed or an existing network (e.g., under 
contract with a testing service) will be utilized to make it convenient for technicians to take 
the exam. Test sites will be selected to minimize the need for long-distance travel or an 
overnight stay away from home. The system will build in the flexibility to add new test sites 
where and as they are needed. Many certifying organizations use hotel conference facilities 
and ballrooms or school classrooms and auditoriums as test sites, and the program will 
investigate this option. Such temporary sites provide flexibility and cost economies and work 
well as long as security needs can be met. Test scheduling also will be designed for the 
convenience of examinees, with tests given during evening hours or on weekends to 
minimize time off from work. Consideration also will be given to computer-based testing 
that would be available to examinees at any time and location. 

Experienced proctors will monitor the examinations, answering procedural questions 
and watching for possible cheating. They most likely will be retained through a professional 
testing service or similar resource organization and will be hired on a temporary basis for 
each administration of the exam. So that they cannot inadvertently help test takers with 
specific questions, proctors unfamiliar with the test material and subject matter will be 
selected. On the other hand, proctors will be given training in any computer-based 
simulations to be used in the exam so that they can demonstrate the process and answer 
process-related questions for examinees. 

3. Test Guides and Other Preparation Strategies 

Certification test scores should accurately reflect examinees’ technical skills, abilities, 
and grasp of fundamentals, rather than any differences in their familiarity with testing 
formats or procedures. Thus, test preparation materials will be developed and distributed to 
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registrants and through education and training programs. Overview materials will describe 
the test administration process, including security procedures at the test site, rest breaks, 
required or provided materials (e.g., scratch paper, pencils), etc. They will describe the test 
sections, including the number of questions in each, and explain the objective of each 
section. A study guide will provide sample questions of the types to be used for each test 
section and explain proven ways to read, think about, and respond to them. It will describe 
the simulation section and explain how the examinee will conduct the simulations. An 
optional resource reference to available study materials may be developed and offered at a 
reasonable fee to help candidate technicians prepare for the exam. Its sale would offset some 
costs of administering the program. These materials will provide as complete information as 
possible to help test takers prepare for the exam and to help them visualize the process so as 
to reduce pre-test anxiety. 

Serious consideration will be given to providing training session for registered 
examinees prior to testing to teach them how to perform any computer-based simulations that 
will be used in the exam. 

4. Scoring 

Pass/fail levels will be set for the initial set of test forms after the first administration. 
Subsequently, pass/fail levels will be set to equate each new exam with previously 
administered tests. Moreover, within a single test administration, the test results from each 
of the several test forms will be normalized across the various test versions. Experienced 
test developers know how to set up test versions to simplify normalization and assure that all 
test takers’ scores are comparable regardless of the test version taken. 

Once tests are scored and equated, the examinees will be notified of their individual 
and overall pass/fail scores and certificates will be issued to those who have passed. Test 
scores, demographics, and other pertinent information will be forwarded to the data base 
manager for input and storage. The industry has said that it wants a list of certified 
technicians. The program will decide whether and in what ways the names of successful 
examinees will be released and/or disseminated and about the release of information upon the 
request of certified technicians. Regardless of process, test results (scores) will be released 
only upon the written request or approval of individuals who took the test and only to people 
they specifically identify. Test results will never be released over the telephone. Every 
release will be documented in writing. 



Schools will be interested in the scores of their students as a measure of the value of 
their training programs. Schools requesting information will be given an averaged combined 
score for their students, if five or more of their students took the exam during a single 
administration. A combined score could reveal the averaged percentage score for all students 
from the requesting school on each of the technical sections and for each of the two overall 
exams; the students’ composite pass/fail score for the six subsection s and tlie two tests; the 
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number of students taking the exam and passing the exam; or the percentage of students 
passing. No information about individual students will be provided. 

Any information released and any presentation of scores will make clear that passage 
of the test and subsequent certification carries specific and limited value: It attests that on 
the day noted the individual thus certified successfully demonstrated technical skills and 
knowledge based on the national skill standards for entry-level, work-ready electronics 
technicians; that all reasonable efforts were made to assure that the person named on the 
certificate is the person who took the test; and that the test was given under controlled 
conditions. Certification cannot be assumed to mean other than that nor to be useful in any 
other context. 



5. Security of the Test and Test Results 

Security measures to protect the integrity of the test will begin at the question writing 
sessions to develop the bank of test questions. Participants will be admonished not to discuss 
or otherwise disclose any body of information developed during this process. All resource 
materials, any notes taken, and any other written materials will be collected by the session 
leader at the end of each meeting and at the completion of the question writing exercise. The 
project manager and the contractor will institute strict controls and utilize locked files and 
facilities and secured computer data bases to safeguard all test question materials during their 
development. Access to the materials used and produced by the test writing groups and to 
the draft questions themselves will be limited to those responsible for development of the 
certification test forms. Tests and testing procedures used during the validation process will 
be subjected to the same security measures as the validated final bank of questions, final test 
forms, and administrations of actual tests. 

Access to the test question bank will be limited to those with a legitimate reason for 
access as determined by the project manager. The project will consider encrypting the test 
question bank to protect against accidental compromise and access by computer hackers. 

Measures successfully employed currently by testing organizations will be instituted to 
assure that administration of the test is not compromised by removal of a test booklet from 
the site or by the copying of questions during test administration. For example, to assist in 
tracking individual test forms, each test copy will be individually numbered; proctors will 
match test number and examinee as tests are given out and collected. Test takers will be 
allowed to take only writing implements and pocket calculators into the test area. Their 
portfolios, attache cases, purses, backpacks, etc., will be stored in a controlled location away 
from the test area, and test takers will be denied access to them during the test. 
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The section on development of the bank of test questions noted that three to five 
questions will be developed for each question to be used in any single test. This is done for 
security purposes. Multiple forms of each test will be developed and used at each test site, 
making it useless for one test taker to copy from another. Also, different test forms will be 
used from one administration of the test to another to reduce the value of tests or test 
answers circulated among technicians in spite of security controls. (People with outstanding 
short-term memories have made significant money selling testing information without any 
overt "cheating" or remo'V’l of test documents from the site.) 

C. DATA MANAGEMENT 

Like any business with employees, products, and income, the certification program 
will need to maintain and protect its personnel records, financial records, and data related to 
social security, unemployment insurance, taxes, etc. A support staff will handle these 
administrative functions. The critical management issues, however, relate to the storage, 
protection, retrieval, and use of data relating directly to administration of the certification 
tests: the test question bank; test forms; test registration data, including demographics about 
test takers; individual test scores; and various crossed-match data sets providing profiles of 
various categories of test takers and certified technicians. The project will establish 
appropriate databases as well as policies for the maintenance and security of these records. 
Issues of data management relating to scoring and security already have been covered. The 
data also will be coded and stored in a manner reflecting its possible future interest to 
researchers. That is, individuals may be interested in analyzing the educational or training 
background of examinees who pass vs. those who fail; age or geographic comparisons of 
successful vs. unsuccessful examinees; work experience before testing; or the benefits of 
certification over short and long-term employment, etc. 

D. PROGRAM MANAGEMENT 

Once the bank of test questions and the test forms have been developed, EIF and EIA 
will make decisions regarding management of the program. Based on the literature review 
and assessment of current certification and accreditation programs, several options exist: 

The test can be managed by an existing industry organization, for example ELA; a new test 
management organization can be established independent of existing organizations; or an 
existing organization, independent of the industry trade and service organizations, can assume 
management of the program in order to reduce implementation time and management costs. 

If the last option is chosen, that organization (most likely EIF) can hire staff to operate the 
program; contract with an existing independent testing company to manage the program 
while it provides oversight and guidance; or manage parts of the program but make the bank 
of test questions and test forms available to test administration services or other industry 
organizations for administration of the test under a fee-sharing arrangement. 
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Industry has stated that management of a certification program by EIA or EIF would 
provide credibility and assure the integrity of the test; studies show management by an 
independent organization, such as a foundation, assures the industry that no single sector is 
being favored over another. Contracting with an existing testing organization for 
administration of the test (registration and onsite operation), instantly provides access to an 
established administrative framework, staff, procedures and policies and limits direct costs to 
personnel costs for the program director, data base manager, marketing specialist and clerical 
staff and to marketing expenses. 

III. MARKETING THE CERTIFICATION TESTING PROGRAM 

As critical as the structural process is, marketing ultimately will be the key to the 
program’s success. Certification and use of the test results are voluntary. The quality of the 
test will matter only if certain marketing goals are met. 

• The national skill standards are broadly recognized as the industry’s definitive 
statement of its requirements; 

• The skill standards-based certification program becomes widely known and is 
recognized as a reliable measure of standards-based competency; 

• Employers view certification as a reliable measure of an individual technician’s 
specific strengths and weaknesses and a valuable tool in hiring; 

• New technicians view certification as a significant edge in obtaining 
employment and seek to become certified; and 

• Technical and vocational educators view the ability of their graduates to 
become certified as a valid measure of the school’s success in providing 
meaningful training based on the skill standards. They teach with certification 
in mind (i.e., to the skill standards) and encourage their graduates to seek 
certification. 

As stated by Eric Rice in a draft to Marketing Guidelines to the Project Evaluation 
Guide, a supplement to the Aguire International evaluation of Department of Labor skill 
standards project: 

"It [marketing] is critical to create an awareness of the opportunity associated 
with the use of the standards; critical to the acceptance of the standards within 
the broader community; critical to the promulgation and widespread use, 
including the portability of standards across the country; and critical to the 
sustainability and renewal of a standards system within any given industry." 
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Because no government regulations or industry requirements force industry or 
education to use the skill standards in course preparation or training or to use certification as 
a yardstick in hiring or a measure of success in training, any marketing program will liave to 
sell convincingly the benefits, advantages, and opportunities associated with standards-based 
certification. 

The framework and techniques employed by that marketing program will depend upon 
the actual certification program that is developed and the management options selected. 

Thus, at this point it is sufficient to point out that marketing will play a key role in the 
success or failure of the certification program regardless of the validity of the certification 
exam and will require a significant measure of creative thinking and a substantial investment 
of funding, particularly in the early years of the certification program. 
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